Summary Neural architectures Training neural nets Forward pass: Tensor operations Backward pass: Backpropagation Neural network design: Activation functions Weight initialization Optimizers Neural networks in practice Model selection Early stopping Memorization capacity and information bottleneck L1/L2 regularization Dropout Batch normalization